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Remarks 

Statement of the Substance of the Interview 

Applicants thank the Examiner for granting the telephone interview of 12 December 2005 
between the Examiner, Applicant Bharti Temkin, and Jack Hamilton, Applicants' counsel. 

During the interview, claims 2 and 3 were discussed in the context of the cited art and the 
instant application. No agreement was reached regarding the claims, but some suggestions were 
made regarding how the claims could be improved to address the concerns of the Examiner. The 
amendments above and the remarks below restate much of the content of Applicants' 
presentation, and reflect an effort to address those concerns of the Examiner. 

Status of the Claims 

Claims 2-5 are now pending in the application. Claim 6 has been canceled above, with 
much of the subject matter of that claim now inserted into amended claim 2. No new subject 
matter has been entered through the amendments of claims 2 and 3. The remarks below address 
the claim rejections presented in the final Office Action dated 19 October 2005. 

Claim rejections under 35 U.S.C. §103 

(a) Claims 2-5 stand rejected under 35 U.S.C. § 103(a) as being obvious in light of the 
combined teachings of Dumoulin 1 and Funda 2 . Applicants respectfully disagree and offer the 
following amendments and remarks to clarify the patentable differences between the presently 
claimed invention and the cited art. The remarks of Applicants' previous responses, dated 12 
September 2003, 24 January 2005, and 05 August 2005, are incorporated herein by reference, even 
if not fully recited again below. 

(c) Claim 2, as amended, recites: 

Method of creating a stereoscopic haptic virtual environment, comprising the steps of: 

providing a computer controlling a display device and operatively coupled to a haptic 

1 U.S. Pat. No. 5,526,812 issued 18 June 1996 to Dumouin, et al. 
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device; 

generating via programming executing on the computer a stereoscopic graphics for 
display on the display device by repeatedly performing the steps of: 

rendering a first video image of a scene on stencil at a first position 
simulating the position of a left eye of a user; 

rendering a second video image of the scene off stencil at a second 
position simulating the position of a right eye of the user; 

interlacing the first video image and the second video image; 

shuttering views of the display device while displaying the interlaced first 
and second video images in order to present in quick succession 
the first image to the left eye of the user and the second video 
image to the right eye of the user, each of said video images 
presented to the associated eye while the view of the other eye is 
shuttered; 

generating haptic scene components including a component representing the 
haptic device; 

time-synchronizing the stereoscopic graphics and haptic scene components; 
presenting the synchronized stereoscopic graphics and haptic scene components 

to a user on the display device; and 
providing a force response to the user when collisions among the stereoscopic 

graphics and the haptic device scene component are resolved. 

(d) The Action asserts that Dumoulin teaches all the subject matter of claim 2, except for the 
use of an input device instead of a haptic scene input device. More particularly, it is stated that in 
both Dumoulin embodiments, the computer image must be registered (coincide) with the 
external structures as viewed by operator 350, and that initialization may be accomplished by 
manual input from the operator to rotate, translate and scale the computer generated image(s) 
until they coincide with the scene observed through the semi-transparent screen, or by employing 



2 U.S. Pat. No. 5,749,362 issued 12 May 1998 to Funda, et al. 
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tracking device 50 to set initial parameters. Further, it is asserted that once the 3D model and the 
visual image of patient 1 are aligned, tracking device 50 keeps the view angles and field of view 
consistent, allowing real-time interactive synchronization between the operator's view of patient 
1 and the computer generated image(s) defined by a stereoscopic viewer (252) (Fig. 2, 
stereoscopic graphics) and the workstation view input device (60) (Fig. 2, scene components.) 

Applicants respectfully submit that the references do not teach or suggest the presently 
claimed invention as alleged in the Action. The synchronizing process addressed by the 
references involves aligning the mono or stereoscopic rendered image of the computer generated 
3D model of the internal structures with the visual image of patient 1 (i.e. the observable real 
external structures) through the semi-transparent screen. A tracking device is used to maintain 
the real-time interactive synchronization between the two, by keeping track of the view angle 
and the field of view. The synchronization processes taught by the cited references and the 
presently claimed invention are different mechanisms. The synchronizing stereoscopic graphics 
and haptic scene components limitation recited in claim 2 ensures that the optical and haptic 
presentations of 3D models are consistent and synchronized (see instant specification, at page 8, 
lines 3-6.) Dumoulin's process of using haptic to register two images is not analogous to 
generating haptic scene components that permit a user to navigate through a computer generated 
virtual environment and touch objects with force-feedback. The references do not teach the 
creation of the necessary 3D geometry or shuttering technique as recited in claim 2. 

The cited references disclose generating pairs of images, for hidden internal objects, and 
synchronizing them to create the illusion of depth. They also teach a method of displaying and 
registering the properly scaled and oriented image of internal objects, including a medical 
instrument inside a patient, with an image of the external visible objects. These images are 
created from an appropriate required vantage point and marked with the location of registration. 
Each reference is limited to using an input device that vibrates when these two positions are 
registered. 
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Dumoulin discloses a method of: (a) tracking and computing position and orientation 
information for the location and orientation angles of left and right eye of the user, patient, semi- 
transparent screen, and invasive device; (b) generating and synchronizing pair of images for the 
internal objects for viewing; (c) scaling and orienting these images to "fit" the view of the 
external objects; and (d) registering the generated image with the image of the external object. 

Per Dumoulin, registration of two images is accomplished either manually or with a 
tracking device, (col. 6, 11. 6-27) Thus, the problem of collision detection between 3D objects and 
a haptic device object does not arise, which only makes sense since the functionality of 
Dumoulin's input device is clearly stated (in three places 3 ) to be other than haptic. This implies 
that even the concept of haptic functionality (touching any point on a surface of a 3D object to 
feel the shape and texture/stiffness, etc) was not being contemplated and thus Dumolin patent 
could not possibly teach generating haptic scene component. Further, since Dumoulin does not 
disclose generating haptic scene components, it logically cannot also disclose synchronization of 
graphics and haptic components. 

Funda does not provide the disclosure missing from Dumoulin. Funda is directed to a 
method of using apparatus hardware, such as images from a video camera, to determine position 
information of an anatomical structure from a gaze direction 4 . Funda clearly states that the 
location/position of an object (designated anatomical feature) is one specifically selected point 
within an image or a volume of the stereoscopic image 5 . Funda's technique of determining an 
object's position by marking a point on an image or a volume containing the object is not 
analogous to permitting touching or working with all the points belonging to the surface of the 
geometrically modeled object. Thus, objects are represented as single points in Funda's 
approach, and no disclosure is provided regarding the geometrical or the topological information 
of the objects 6 . Funda's single point-based positional information is used to register the external 



3 Col. 4, 11. 37-41; col. 6, II. 28-32; col. 7, 11. 7-8 

4 Col. 3, line 48 to col. , line 2; col. 4, line 30 to col. 5, line 63 

5 Col. 7, 11. 27-67 

6 Col. 9, 11. 13-22; col. 11,11. 10-20 
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and internal objects (or a medical instrument position). When the two points coincide, tactile 
vibrations are felt. 

Clearly, neither Dumoulin and Funda use 3D geometrical information for registration and 
neither contemplated creating scene graphs or developing the software structures needed to 
represent the geometry of each object. Neither of the patents teach the functionality of generating 
graphics or haptics scene component. In fact the fundamental concept of scene graph does not 
even occur in either of the patent. 

As described in the instant specification, the location, 3D shape, and feel of the object are 
programmed so that, when the user moves in the 3D work space, the interface device positional 
information in virtual space is retained, and based on what the object being touched (as 
programmed) the device provides a force feedback to the hand of the user. The cited references 
permit viewing of object shapes, but said objects are not programmed to be touchable. As the 
haptic device is moved, the touching of the objects in the scene is determined by resolving 
collisions with the virtual objects,a nd depending on the type of object encountered, the interface 
device presents the user with force-feedback calculated to duplicate the force that an actual 
object would provide. 3D objects are modeled and have geometrical representations that provide 
shapes and topologies of modeled objects. 3D stereoscopic display methods taught by the cited 
references do not teach this critical component that is essential to make these object touchable by 
a haptic device; they provide methods for generating two images in order to create the 3D 
stereoscopic display that produces illusion of the depth perception. 

Transformational content (for translation, rotation, scaling, etc) includes relational 
information between the individual modeled object's local coordinate system and a shared world 
coordinate system of the whole VE. This information makes it possible to independently orient 
and position the individual objects within the 3D virtual environment. 3D stereoscopic display 
methods taught by the cited references do not teach this critical component that is essential to 
make sure that the haptic device moving in the virtual environment act independently of the 
environment; 3D stereoscopic display taught by the cited references is capable of scaling and 
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rotational transformations, however, since objects within do not have geometrical representation 
they do not teach the art of creating object's local coordinate system, and thus obviously relating 
it to the world coordinate system. Furthermore, haptics scene components also include the 3D 
haptic device virtual representation component. This component representation (proxy) of the 
haptic device in the virtual environment's 3D space is comprised of a 3D geometrical object that 
also has information (such as 6 degrees of freedom (DOF) device position/orientation and 3- 
DOF of force feedback) that is used to link the proxy with the actual haptic device. 3D 
stereoscopic display methods taught by the cited references do not teach this critical component 
that is essential to provide visual cue (as recited in amended claim 2) of the navigation position 
through the virtual environment while using a haptic device (moving within the geometrically 
modeled virtual environment) to touch objects. 

The hierarchical software structure designed for the presently claimed invention's 
stereoscopic graphics and haptics scene components make it possible to create 3D haptic 
interactions between: (i) the haptic device moving in the real work space (this is the actual empty 
space in which the haptic device moves); (ii) the proxy; and (iii) virtual objects in the virtual 
work space - the 3D virtual environment. During these interactions, a collision occurs when the 
proxy touches a virtual 3D geometrical object. When a collision occurs, the physical properties 
of the geometrical object are used to compute a force feedback. The computed force is sent to the 
haptic device (via the link to the proxy). This force response provides the sense of touch 
resulting in being able to tell the shape, texture, and compliance of the object. This force 
response can also be used to manipulate the virtual objects (e.g. the objects can be pushed and 
the displacement of the object will be directly related to the force used to push). 3D stereoscopic 
display methods taught by the cited references do not teach this critical component that is 
essential to feel the objects being touched in the virtual environment and feel their shapes and 
textures, compliance, etc. This is not possible to achieve with the methods disclosed in the cited 
references. 

Dumoulin and Funda patents elaborate on generating and representing pairs of images 
that allow creation of an illusion of the depth perception (making it 3D) needed for their 
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stereoscopic displays, focusing on computing point positions where the registration of the two 
images occur. Such methodologies cannot be used to create a true 3D computational 
environment, such as recited in amended claim 2. 

In light of at least the foregoing, Applicants respectfully submit that claims 2-5, as 
amended, are patentable over the cited references, and respectfully request reconsideration and 
withdrawal of all grounds for rejecting the claims and objections to the specification. Applicant 
earnestly solicit a notice regarding the allowability of the claims. 

The Examiner is invited to call Applicants' counsel at (617) 854-4000 at the time of 
review of this amendment, in order to have answered any remaining questions and to avoid the 
expense of another round of written response. 

Respectfully submitted, 

BHARTI TEMKIN, et al., Applicants 



Date: February 10, 2006 




Reg. No. 48,946 
Attorney for Applicants 
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